Skip to content

QA Video Automation: Recording, Finalization & File-backed Attachments#6935

Closed
Jasonnnz wants to merge 74 commits into
mainfrom
feature/qa-video-automation
Closed

QA Video Automation: Recording, Finalization & File-backed Attachments#6935
Jasonnnz wants to merge 74 commits into
mainfrom
feature/qa-video-automation

Conversation

@Jasonnnz
Copy link
Copy Markdown
Contributor

@Jasonnnz Jasonnnz commented Feb 23, 2026

Summary

Enables Velly to run QA/test workflows through computer-use sessions, record the screen, and return a playable + draggable video in chat. Adds QA intent detection, IPC contract extensions, daemon-side finalization handling with summary + attachment injection, macOS screen recording lifecycle, file-backed attachment storage with HTTP range-request streaming, chat video playback with drag-to-Finder support, and configurable retention cleanup.

Changes

  • QA Intent Detection: detectQaIntent() pattern-matches QA/test commands and natural-language requests (narrowed to avoid false positives on common words like "check")
  • IPC Contract Extensions: reportToSessionId, qaMode on CuSessionCreate and TaskRouted; conversationId on TaskSubmit; retentionDays, captureScope, includeAudio on TaskRouted; new CuSessionFinalized message with recording metadata
  • Daemon Routing: Task routing and escalation handlers detect QA intent, set qaMode + reportToSessionId, pass retention/capture config
  • Client QA Wiring: AppDelegate+Sessions.swift reads qaMode from task_routed, creates ScreenRecorder, passes QA params + retention + capture config to ComputerUseSession; sends conversationId in task_submit
  • Daemon Finalization Handler: handleCuSessionFinalized creates file-backed attachment (always, for cleanup), injects summary + attachment into source chat (when reportToSessionId present). Uses recordingTracked flag to catch all edge cases.
  • File-backed Attachment Schema: New storageKind, filePath, sha256, expiresAt columns; createFileBackedAttachment() for large files; backward-compatible
  • Content Streaming Endpoint: GET /v1/attachments/:id/content with RFC 7233 range request support
  • macOS Recording Lifecycle: ScreenRecorder using ScreenCaptureKit; integrated into Session.swift; configurable retention + capture scope + audio from daemon config
  • Capture Scope Resolution: When captureScope is "window", resolves topmost non-self CGWindowID (excludes own PID + filters to layer 0); when "display", passes CGMainDisplayID(). Falls back gracefully.
  • Chat Video UX: InlineVideoAttachmentView fetches from /content for file-backed attachments; .onDrag for drag-to-Finder
  • Retention Cleanup: Configurable qaRecording.defaultRetentionDays; background cleanup task; orphan recordings tracked in DB
  • Cancel Safety: Deferred abort with safety-net timer; timer properly disarmed via Task.isCancelled guard
  • Metadata Lifecycle: Socket close handler owns socketToCuSession cleanup; targeted cleanup on session replacement
  • Recording Fallback: recordingTracked flag ensures all untracked recordings get DB entries (missing conversation, empty summary, etc.)

Milestone PRs

Fix PRs

Known limitations (deferred to M3/M4)

  • QA intent detection is regex-based — Works for direct commands but misses nuanced requests ("smoke test onboarding"). LLM-based semantic classification planned for M3.
  • Capture config is daemon-config-only, not per-requestcaptureScope and includeAudio always come from daemon config. Per-request overrides planned for M3.
  • Chat injection requires conversationId — If absent on task_submit, recording is tracked for cleanup but not injected into chat. Product decision deferred.
  • Saved QA scripts (M3) and scheduled self-QA (M4) remain unimplemented.

Project issue

Closes #6899

Test plan

  • Verify IPC contract snapshot test passes
  • Verify file-backed attachment CRUD tests pass
  • Verify attachment content route tests (range requests, 206/416 responses)
  • Verify recording cleanup tests
  • Verify cu_session_finalized handler tests (7 tests including edge cases)
  • Manual: send "test the onboarding flow" and verify QA intent triggers recording
  • Manual: send "check my email" and verify QA mode does NOT trigger
  • Manual: verify recording appears as playable video in chat with drag support
  • Manual: verify retention cleanup removes expired recordings
  • Manual: set qaRecording.captureScope to "window" and verify target app window is captured (not Vellum)

Generated with Claude Code

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

QA Video Automation: Ad Hoc Recording + File-backed Attachments

1 participant